Entropy and Relative Entropy From Information-Theoretic Principles

We introduce an axiomatic approach to entropies and relative that relies only on minimal information-theoretic axioms, namely monotonicity under mixing data-processing as well additivity for product distributions. find these axioms induce sufficient structure establish continuity in the interior of probability simplex meaningful upper lower bounds, e.g., we every entropy satisfying must lie between Rényi divergences order 0 ∞. further show simple conditions positive definiteness such a characterisation terms variant trumping. Our main result is one-to-one correspondence entropies.